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(54) RALLY GAME SYSTEM USING VIEW LABEL 

(57)Abstract: 

PROBLEM TO BE SOLVED: To present computer- 
derived topographic information and an image showing 
real view to each user while matching the topographic 
information with each part of the image and to make an 
automobile rally game possible in a real city. 
SOLUTION: This system transmits position information 
(with time) about the automobiles of users and 
automobile information from a terminal 10 to a center 20 
at regular time intervals. A game information control part 
25 controls the position information about each 
automobile and the automobile information, and delivers 
the position information about the users and order 
information to a label information originating part 22. The 
label information originating part 22 originates rally order 

label information such that rally order information corresponds to an automobile area in an 
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image of the view. The rally order label information is delivered to the label information output 
part 17 of the terminal 10 and displayed on the windshield of the automobile of each user. 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the lnvention]This invention carries out the superimposed display of the geographical 
information about each subregion in the picture to an image display device to the picture which 
the user photoed using landscape image input devices, such as a camera, or relates to the 
device which carries out an audio assist etc. and is taught to a user. 
[0002] 

[Description of the Prior Art]Conventionally, there were various navigation systems as a 
system which teaches a user the geographical information about the circumference in which a 
user is present. 

[0003] Drawing 12 is a lineblock diagram of the navigation device indicated by JP 8-273000.A 
The position updating part 71 which will update the position of vehicles with reference to road 
map data if this device inputs the position data and motion data of vehicles, The data 
generating part 72 for a display which generates the road data for a display, and the 
background data for a display based on map data etc., The three-dimensional dynamic-image- 
data preparing part 73 which creates three-dimensional dynamic image data based on these 
data for a display, It has the storage parts store 74, and when the user of a navigation device 
sets up a priori a running path including the destination and a course place, it has a function in 
which a course can be set up, looking at the real cine mode display screen along the road 
which actually exists instead of a map screen. 

[0004]According to this device, the user can see the cine mode display (for example, drawing 
13) in alignment with that course, when running in accordance with the course which actually 
exists. 
[0005] 

[Problem(s) to be Solved by the lnvention]However, when using the device, and human being 
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matches actual scenery and the geographical information in the world of a computer eventually 
with the naked eye, it must be recognized what the thing in actual scenery is. That is, based on 
the sign in the map with which the cine mode display of what the actual building and road in a 
user's view, and a mountain are was carried out, etc., a naked eye is made reliance, human 
being's brain is used unconsciously, and human being has to work matching and has to 
understand. If the map in a computer and the actual scene were compared in the street corner, 
the direction has been grasped, or the mark was found and it gazed at the direction, after 
understanding the feature of the building which exists in the direction, the map was seen 
again, and he understands what the building is. 

[0006]For this reason, the time and effort which compares the map and real landscape on a 
computer repeatedly, and is matched at human beings has the problem of not being omissible. 
A real landscape is hard to see and cannot take correspondence easily especially in 
semidarkness, the night, etc. 

[0007]There is a fault that the automobile rally game in the inside of a personal computer has 
few senses of reality, and they are unsatisfactory. 

[0008]The purpose of this invention is to provide the scene label use type rally game system 
which can do a rally game in the actual street while matching each portion in the geographical 
information on a computer, and the picture (it is hereafter called a landscape image.) of a real 
landscape and teaching a user. 
[0009] 

[Means for Solving the Problem]This invention creates map data on a computer beforehand as 
three-dimensional data, A position in case a picture (it is henceforth called a landscape image 
in order to distinguish from a CG image) is inputted, an angle of a camera, a focal distance, 
and image size are acquired at the time of photography, Computer graphics at the time of 
viewing from a position at the time of real landscape photography, an angle of a camera, and a 
focal distance in three-dimensional map space on a computer. (It is hereafter referred to as 
CG.) Geographical information within a picture is acquired and matching is realized by carrying 
out the superimposed display of the geographical information to a landscape image which is a 
real landscape. This geographical information is names, such as a structure, or attribution 
information of those in a picture, and attribution information means information about all 
attributes (for example, an outline, a color, etc.) about that structure. Suppose that the word 
"structure" is used in the sense of all data that has a certain geographical structures in the map 
DB also including natural geographical features, such as a mountain, a river, and the sea, in 
addition to an artificial structure in this specification. In acquisition of geographical information, 
it asks for a landscape image based on a camera position, a camera angle, a focal distance, 
and image size, and asks for a structure of a multiple image. A position (a grant position is 
called hereafter) of a landscape image in which the structure should be reflected is searched 
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for, and the superimposed display of a name or attribution information of a structure is carried 
out. 

[0010]ln order to raise further accuracy of matching with a structure in a landscape image, and 
a structure in a CG image, a structure previously gained to each subregion of a landscape 
image is matched with pattern matching. A CG image is created based on a gained structure, 
and it asks for a structure which matched subregion in a CG image with pattern matching to 
said subregion of a landscape image, and became a basis of matched subregion. 
[001 1]Here, an example of a method of creating a CG image is described. The three- 
dimensional map DB is accessed based on a camera position, a camera angle, a focal 
distance, and image size which were acquired previously, and it asks for view space in three- 
dimensional map space. It asks for a structure in view space, and three-dimensional projection 
conversion of the solid data of each structure is carried out to this surface of projection by 
using a camera screen as a surface of projection. It hides in other structures among line data 
which furthermore constitutes a projection form of each structure, and the hidden line removal 
of the hidden-line data is carried out using techniques, such as a normal vector method. A 
hidden line removal is carried out and area division of the CG image is carried out based on 
the remaining line data. Since the three-dimensional map DB is used, a name of a structure 
which serves as a basis of the field for every field can be matched. 

[0012]Then, a structure name of subregion of a CG image matched with each subregion of a 
landscape image by pattern matching is extracted. Three-dimensional projection conversion of 
the position coordinate of a structure in inside of three-dimensional map space is carried out to 
a previous surface of projection, and a position coordinate of a real landscape picture which 
should superimpose an extracted structure name is searched for. Label information is created 
from a position coordinate of a real landscape picture which should superimpose an extracted 
structure name. A structure name is superimposed on a landscape image which is a real 
landscape based on label information, and it displays on vision apparatus. 
[0013]ln this invention, in order to make an automobile rally game in the still more nearly actual 
street, the following composition is taken. 

[001 4]A communication control means sends position information (with time) and car 
information about a user to a center for every fixed time from a terminal. Here, having called it 
car information consists of attribution information (for example, a registration number of a car, 
the vehicle name / model / color of a car, using frequency of a transceiver of automobile 
loading, etc.) about a car which each user drives. A communication control means by the side 
of a center passes position information and car information about a user to a game information 
management tool via a center control means. In a game information management tool, each 
user's position information and car information are managed, precedence information in a rally 
of each car computed from position information about a car which a user who participates in a 
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game drives, and position information and map information is managed, and position 
information and precedence information are passed to a center control means. In a center 
control means, car information is passed to a user's position information and a precedence 
information pan at a label information preparing means. In a label information preparing 
means, based on a user's position information and car information under intervention, When a 
car which exists in a landscape image is recognized and a registration number of the car has 
been recognized from a number plate of the car, If it can presume that the registration number 
is a registration number of a car which a certain user drives from a point of position information 
and car information, rally ranking label information which made precedence information of a 
rally correspond to a field of a car in a landscape image will be created, and, in addition to the 
usual label information, a center control means will be passed. Here, it becomes rally ranking 
label information from attribution information including a name of an automobile slack structure 
which a user under intervention operates, and precedence information, and its position 
information. Via a communication control part, over a label information outputting part, rally 
ranking label information superimposes a ranking number on a picture of a car, and is 
displayed on a windshield of a user's car. 

[001 5]A scene label use type rally game system of this invention, Consist of two or more 
terminals and centers which were carried in a car, respectively, and said each terminal, An 
image acquiring means which acquires a picture, and a position information acquiring means 
which acquires a camera position at the time of image acquisition, A camera attribution 
information acquisition means which acquires a camera angle, a focal distance, and image 
size at the time of image acquisition, An image processing means which divides an acquired 
picture into two or more subregions, and information about area division, said camera position, 
said camera angle, said focal distance and said image size of said picture via a 
communications network, A communication control means which transmits to a center, 
transmits to a center car information which is position information with time on a car of the 
terminal concerned, and the attribution information of a car via a communications network for 
every fixed time, and receives label information and rally ranking label information from said 
center, A name of a structure in said label information or its attribution information is 
superimposed on a position corresponding to a grant position in a picture, Have a label 
information output means which displays a picture on which it was superimposed on vision 
apparatus, and displays said rally ranking label information on a windshield of a car, and a 
terminal-control means to control said each means, and said center, Information about area 
division of said picture, said camera position, said camera angle, said focal distance, said 
image size, position information with time on each terminal, and car information are received 
from said each terminal via said communications network, A communication control means 
which transmits said label information and said rally ranking label information to said terminal, 
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A map information management tool which gains a structure which manages map information, 
asks for view space in map information space based on a camera position, a camera angle, a 
focal distance, and image size which were received, and exists all over the view space, A 
game information management tool which manages position information with time and car 
information of each terminal, and manages precedence information in a rally of each car which 
a user who participates in an automobile rally game drives, Said label information which 
matches said gained structure to said subregion of said picture, and includes a name or 
attribution information, and a grant position of said structure which were matched is created, 
Based on a user's position information and car information under rally game intervention, 
When a registration number of the car has been recognized from a number plate of the car and 
the registration number can presume a car which exists in a landscape image to be a 
registration number of a car which a certain user drives from a point of position information and 
car information, It has a label information preparing means which creates rally ranking label 
information which made precedence information of a rally game correspond to a field of a car 
in a landscape image, and a center control means which controls said each means. 
[0016]According to the embodiment of this invention, a label information preparing means, A 
CG image which is a computer graphics image is created based on a gained structure, 
Subregion in said CG image is matched with pattern matching to said subregion of said 
picture, it asks for a structure of matched subregion, and label information including a name or 
attribution information, and a grant position of the structure is created. 
[0017]According to other embodiments of this invention, a label information preparing means, 
Carry out three-dimensional projection conversion of the gained structure to a camera screen, 
eliminate a structure which does not appear from a viewpoint, and a CG image is created, A 
border line of subregion in a CG image divides a CG image into subregion, It asks for a 
structure which matched said subregion of said picture, and said subregion of said CG image 
with pattern matching, matched to subregion of a picture, and became a basis of subregion of 
a CG image, and label information including a name or attribution information, and a grant 
position of the structure is created. 
[0018] 

[Embodiment of the lnvention]Next, the embodiment of this invention is described with 
reference to drawings. 

[00 19] Reference of drawing 1 will constitute the scene label use type rally game system of one 
embodiment of this invention from the terminal 10, the center 20, and the communications 
network 30 which were carried in the car. Here, two or more terminals 10 exist actually, 
although the deer graphic display even of the **** top is not carried out. 
[0020]The position information acquisition part 12 which acquires the image acquiring part 1 1 
from which the terminal 10 acquires a picture, and the camera position at the time of image 
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acquisition, and the camera attribution information acquisition part 13 which acquires the 
camera angle, the focal distance, and image size at the time of image acquisition, The image 
processing portion 14 which divides the acquired picture into two or more subregions, and the 
information and camera position about area division, the camera angle, the focal distance and 
image size of said picture are transmitted to the center 20 via a communications network, the 
attribution information (the registration number of a car.) about the position information with 
time on a car and the car 10 of the terminal 10 The car information which consists of using 
frequency of the transceiver of a vehicle name / model / color, and automobile loading, etc. is 
transmitted to the center 20 via the communications network 30 for every fixed time, The 
communication control part 15 which receives label information and rally ranking label 
information from the center 20, The label information outputting part 17 which superimposes 
the name of the structure in label information, or its attribution information on the position 
corresponding to the grant position in a picture, and displays the picture on which it was 
superimposed on vision apparatus, and displays rally ranking label information on the 
windshield of a car, It comprises the terminal control section 16 which controls each part 11- 
15, and 17. 

[0021]The center 20 receives the information and camera position about the area division of a 
picture, a camera angle, a focal distance, image size, the position information with time on 
each terminal, and car information from each terminal 10 via the communications network 30, 
The communication control part 23 which transmits label information and rally ranking label 
information to the terminal 10, The map information Management Department 21 which gains 
the structure which manages map information, asks for view space in map information space 
based on the camera position, the camera angle, the focal distance, and image size which 
were received, and exists all over the view space, The game information Management 
Department 25 which manages the position information with time and car information of each 
terminal 10, and manages the precedence information in the rally of each car which the user 
who participates in an automobile rally game drives, The label information which matches the 
structure gained to the subregion of said picture, and includes the name or the attribution 
information, and the grant position of a structure which were matched is created, Based on a 
user's position information and car information under rally game intervention, When the 
registration number of the car has been recognized from the number plate of the car and the 
registration number can presume the car which exists in a landscape image to be a registration 
number of the car which a certain user drives from a point of position information and car 
information, It comprises the label information preparing part 22 which creates the rally ranking 
label information which made the precedence information of the rally game superimpose on 
the field of the car in a landscape image, and the center control section 24 which controls each 
part 21-23, and 25. 
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[0022]Next, operation of this embodiment is explained with reference to the flow chart of 

drawing 2 . 

[0023]First, in order that the terminal control section 16 may acquire the information about a 
landscape image, a processing start command is sent to the position information acquisition 
part 12, the camera attribution information acquisition part 13, and the image acquiring part 11. 
The position information acquisition part 12 collects position information per second by a GPS 
receiver etc. in response to a command from the terminal control section 16, and passes it to 
the terminal control section 16 (Step 21). Here, a time interval is good not only for how [ a 
second bit but ]. The camera attribution information acquisition part 13 acquires the camera 
angle of landscape image recorders, such as a camera at the time of picture photography, in 
the group of a horizontal angle and an ascending vertical angle in response to the command of 
the terminal control section 16 (Step 22), and if it is a landscape image device which has a 
zoom function simultaneously, it will acquire a focal distance (Step 23). The image acquiring 
part 1 1 acquires a landscape image per second from the terminal control section 16 in 
response to a command, and passes it to the control section 16 (Step 24). Since image size is 
immobilization for every landscape image device, the terminal control section 16 holds image 
size information. The terminal control section 16 holds the collected information as a 
landscape image file. 

[0024] Drawing 3 shows the file format of the data structure of a landscape image file. A 
landscape image file has header information and image data. As header information, it has 
position information, camera angle information, a focal distance, time information, the image 
size of a graphics file, a type, and size. As position information, it has each data (for example, 
east longitude 137-degree 55 minutes and 10 seconds, north latitude 34-degree 30 seconds of 
34 minutes, altitude of 101 m 33 cm, etc.) of the east longitude, the north latitude, and altitude. 
As a camera angle, it has each data (for example, 254 horizontal-angle right-handed rotations, 
15 ascending vertical angles, etc.) of a horizontal angle and an ascending vertical angle. Focal 
distance data is the focal distances (for example, 28 mm etc.) of the camera lens at the time of 
picture photography. As time information, it has the time at the time of photography (for 
example, Japan Standard Time 15:06 17 etc. seconds etc. on January 31, 1997). As image 
size of a graphics file, it has pixel size (for example, 640x480 grade) in every direction. 
Similarly it has file types (TIFE form, an 8-bit color, etc.). Similarly it has the numbers of bytes 
(307.2 KB etc.) of a file. It has the image data itself in a binary format. 
[0025]lf the terminal control section 16 stores a landscape image file, to the image processing 
portion 14, it will extract a border line from a landscape image, and will order to divide a 
landscape image into two or more fields. In the image processing portion 14, if it says roughly, 
a differential process will be performed based on the density difference in a landscape image, 
a border line will be extracted (Step 25), and area division is carried out by performing labeling 



http://www4.ipdl. inpit.go.ip/cgi-bin/tran_web_cgi_eiie?atw_u=http%3A%2F%2Fwww4.ip... 4/15/2009 



JP,11-057209,A [DETAILED DESCRIPTION] 



Page 8 of 14 



bordering on the border line (Step 26). The technical term called labeling used here is a 
technical term used in the area division of a picture. As a procedure, a picture is first changed 
into monochrome shade image. Since an outline is a portion into which a luminosity changes 
suddenly, a border line is extracted by performing a differential process and asking for a 
portion with a larger differential value than a threshold. At this time, the line width of a border 
line is 1 pixel, and the border line is connected. Therefore, thinning is performed and the 
connected line with a line width of 1 pixel is obtained. A differential process and thinning are 
enough here, if a certain technique is used from the former. 

[0026]The obtained border line is considered to be a border line of a field, and operation of 
numbering the field constituted by a border line is performed. The maximum number in the 
number turns into the number of fields, and the pixel number in a field expresses the area of 
the field. The example which divided the landscape image into two or more subregions is 
shown in drawing 8. The measure of the similarity (nearness) between fields may be 
introduced and clustering processing which summarizes two or more fields in which character 
is alike to one field may be performed. It is good also by the clustering method like the existing 
method throat. 

[0027]lf area division processing of a landscape image is completed, the communication 
control part 15 will transmit the information (header information of a landscape image file) 
about the area division of the acquired camera position, a camera angle, a focal distance, 
image size, and a picture to the center 20 with directions of the terminal control section 16 
(Step 27). These information is passed to the map information Management Department 21 of 
the center 20 through the communication control part 23 and the center control section 24, and 
the map information Management Department 21 accesses the map DB, and performs 
calculation processing of view space (Step 28). There is a map database program as an 
example of the map information Management Department 21. The map information 
Management Department 21 has managed three-dimensional map data. Although two- 
dimensional map data may be sufficient, since there is no height information in that case, the 
accuracy of the grant position of the labeling to a real landscape is inferior. Height information 
is compensated and processed when carrying out based on two-dimensional map data. For 
example, if there is number-of-stories information as which a house expresses several stories 
when it is two dimensional data of a house, three-dimensional data will be created based on 
the height information which multiplied the number of stories by fixed numbers, presumed the 
height of the house, was presumed to be two dimensional data, and was searched for. Even 
when there is no number-of-stories information, height information can be presumed by 
assigning a fixed number of height according to the area of a house figure, and three- 
dimensional data is similarly created based on presumed height information. In this way, three- 
dimensional data is created and processing is advanced. 
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[0028]The example of three-dimensional map data is shown in drawing 4 . The map information 
space expressed by two dimensions is shown in drawing 4 (1), and the map information space 
expressed by the three dimension is shown in drawin g 4 (2). In response to the command of 
the center control section 24, view space is computed based on the header information of a 
landscape image file to this three-dimensional map information space at the map information 
Management Department 21 (Step 28). The example computation of view space is shown in 
drawing 5 . First, XY axis shall stretch horizontally and the Z-axis shall stretch perpendicularly. 
From the position information in the header information of a landscape image file, the position 
of the viewpoint E is set up in three-dimensional map information space. For example, if it is 
east longitude 137-degree 55 minutes and 19 seconds, and north latitude 34-degree 30 
seconds of 34 minutes and the altitude of 101 m 33 cm, the corresponding coordinates in the 
map mesh number corresponding to it will be set up. Similarly a camera angular direction is set 
up based on the horizontal angle and ascending vertical angle in the camera angle information 
in header information. The focus F is taken at the point which progressed by the focal distance 
from the viewpoint E on the straight line showing a camera angular direction. A sight line 
direction vector is a unit vector of the length 1 which comes out from the viewpoint E on the 
straight line. The width x in the X-axis of a camera screen is set up from lateral size with the 
image size of a landscape image file, and the width y in a Y-axis is set up from the size of a 
lengthwise direction. To a sight line direction vector, the flat surface of the horizontal x length y 
is vertical to a camera angular direction, and it is set up so that the focus F may be included. It 
asks for the straight line which connects the point of four corners of a camera screen from the 
coordinates of the viewpoint E respectively, and let three-dimensional space which four half 
lines prolonged from the viewpoint E make be view space. The example of the view space in 
three-dimensional map space is shown in drawing 6. It looks at three-dimensional map space 
from XZ flat surface. The portion surrounded with the slash in drawing 6 is a sectional view in 
XZ flat surface of the space belonging to view space. The building and mountain in view space 
are contained in the example of drawing 6 . 

[0029]lt asks for the structure which exists in the view space for which it asked at the map 
information Management Department 21. Each peak which constitutes the solid showing a 
structure calculates whether it exists in the interior area of view space for every structure. 
Usually, two-dimensional map space is divided in the two-dimensional mesh of certain size. In 
addition to the mesh of the direction of two dimensions in every direction, as how to cut the 
mesh of three-dimensional map space, a mesh is cut with the constant interval also to the 
height direction. Space will be divided in the unit space of a rectangular parallelepiped. First, 
the existence of a lapped part with view space is investigated the whole unit space of a 
rectangular parallelepiped, and the number of three-dimensional unit map space with a lapped 
part is searched for. The number of three-dimensional unit map space here is the same as that 
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what is called of a mesh number. The existence of view space and a lapped part is 
investigated to the structure in three-dimensional unit map space with a lap. It asks for the 
straight line which connects the coordinates of the peak and the coordinates of a viewpoint 
which constitute a structure, and if the straight line has an intersection to the camera screen of 
drawing 7, it is in view space. If at least the one peak fulfills this condition among two or more 
peaks which constitute a structure, that structure shall have view space and a lapped part. 
[0030]When a structure is contained in the inside of view space or that part is included, the 
processing which carries out three-dimensional projection conversion of each structure to this 
surface of projection is started by using a camera screen as a surface of projection (Step 29). 
Here, as shown in drawing 7, after reexpressing the point P by the coordinate system based 
on the viewpoint E based on a following formula (1), the point P is projected on a camera 
screen and it asks for the intersection Q. 
[0031] 
[Equation 1] 



Coordinate-vectors L= of the coordinate-point E=(ex, ey, ez):viewpoint of the peak which 
constitutes a point P=(x, y, z):structure here (Ix, ly, Iz): Sight line direction vector (unit vector) 



based on the viewpoint E of the point P'=(x', y', z'):point P (X, Y): The projected point t to the 
camera screen of the point P is a focal distance. [0032]ln three-dimensional projection 
conversion, the field which the peak stretches for every structure first is searched for. For 
example, six fields can be found if it is a structure expressed in a rectangular parallelepiped. 
When carrying out projection conversion of each field to a camera screen, to each pixel on the 
camera screen included in a projection area, the distance of a viewpoint and the corresponding 
points on the field is calculated, and it stores in a memory as a depth value (Z value). For 
every field of each structure, the depth value (Z value) over each pixel on a camera screen is 
calculated, and it stores in a memory, z' in one expresses the depth value (Z value) from a 
viewpoint (formula). 

[0033]There are a structure which appears from a viewpoint, and a structure not appearing in 
inside of a structure by which three-dimensional projection conversion was carried out to a 
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camera screen. It is necessary to ask only for a structure which appears from a viewpoint in it, 
and to search for a field located from a viewpoint in an opposite hand, and a field interrupted 
by other structures. Then, hidden surface elimination is performed (Step 30). Although it is in a 
method of hidden surface processing variously, a Z buffer algorithm is used, for example. 
Other scanline algorithms and the ray tracing method may be sufficient. 
[0034]A field which takes a pixel on a camera screen arbitrarily and takes the smallest depth 
value to the pixel is searched for. Thus, if sequential operation is continued about each field of 
each structure, a field nearest to a viewpoint will be left behind for every pixel on a camera 
screen. Since a camera screen top pixel in which a field nearest to a viewpoint is determined 
for every pixel on a camera screen, and a field nearest to a viewpoint is common generally 
constitutes a field, in a camera screen, two or more fields which consist of a pixel which makes 
a common field the nearest field are made. In this way, a field which was able to be found is a 
field of a result of having carried out three-dimensional projection conversion of the subregion 
of a structure which appears from a viewpoint. A field in an opposite hand and a field 
interrupted by other structures are eliminated from a viewpoint. 
[0035]ln this way, a made field forms a CG picture area (Step 31). 

[0036]To apex coordinates of a two-dimensional figure which constitutes a CG picture area, 
three-dimensional coordinates before projection conversion are searched for, and it stores in a 
memory by making both correspondence relation into link information. It is used for asking etc. 
of which structure the two-dimensional field is a projection based on link information. 
[0037]A hidden line removal is carried out and area division of the CG image is carried out 
based on the remaining line data. Since the three-dimensional map DB is used, a name of a 
structure which serves as a basis of the field for every field can be matched. A field to which a 
CG image was divided is numbered in order. An example which divided a CG image into two 
or more subregions is shown in drawing 9. 

[0038]lf area division processing of a CG image is completed, the center control section 24 will 
order to the label information preparing part 22 to match a region division of a CG image, and 
a region division of a landscape image. In the label information preparing part 22, a region 
division of a CG image and a region division of a landscape image are matched by template 
matching (refer to Step 32 and drawing 10). 

[0039]lt matches with a region division of a CG image sequentially from a young field (for 
example, No. 1) of a number among region divisions of a landscape image. In matching, 
although it is very good in which [ of the former to a certain MACHINGU methods ], the simple 
template-matching method is taken here. That is, when you pile up two fields to compare and 
there is a ratio of an overlapping portion more than a fixed ratio fixed as a threshold, suppose 
that it matches as a field about the same structure. For example, a coordinate value of each 
pixel in the field is set to (A, B) about R1 of the 1st region division of a landscape image. 
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Values of a pixel in coordinates (A, B) are an inside of a field, therefore 1. In the 1st region 
division S1 of a CG image, if coordinates (A, B) become in the field S1 , it will be the pixel value 
1 and will lap, but if it becomes outside S1 , it will be the pixel value 0 and will not lap. In this 
way, as the lap coefficient K (A, B) in coordinates (A, B), 1 and when lapping, and not lapping, 
it is decided by 0. Coordinates (A, B) are moved in the field R1, and the lap coefficient K (A, B) 
is calculated. And to several N1 of coordinates (A, B) moved in the field R1, the lap coefficient 
K (A, B) calculates several N2 of coordinates which were 1, and a thing corresponding to a 
case where N1/N2 are more than a threshold, in the region division R1 of a landscape image 
and the region division S1 of a CG image is decided on. This matching is performed from the 
1 st of a region division of a landscape image to the last thing. Even if a position gap of some is 
in an XY direction in addition to this as a matching method, a valuation function which 
becomes the same value may be used. 

[0040]After matching subregion of a CG image to subregion of a landscape image in the label 
information preparing part 22, It is matched, a structure of subregion is extracted (Step 33), 
and processing (Step 34) which searches for information (a name or attribution information of a 
structure) which should be further superimposed for every subregion of a landscape image, 
and is created as label information with a position which should be superimposed is started. 
First, subregion of a corresponding CG image is taken out to subregion of a landscape image. 
Subregion of a taken-out CG image is obtained by carrying out three-dimensional projection 
conversion of the field which has a three-dimensional structure thing in three-dimensional map 
space from the first to a camera screen. Then, a depth value (Z value) in which subregion of a 
CG image has a field of a three-dimensional structure thing used as a basis of three- 
dimensional projection conversion is calculated as a key. When three-dimensional projection 
conversion is carried out previously, created link information may be used as a key. Based on 
a field of a structure used as a basis, the three-dimensional map DB is accessed and a name 
or attribution information of the structure is acquired. It is [ anything ] good if attribution 
information is information which means information which accompanies about the structure 
and starts the structure here. And a position coordinate which should superimpose a name or 
attribution information is decided to subregion of a landscape image. How to determine may 
determine how. For example, the center of figure which stretches subregion may be sufficient. 
Label information is created from a name or attribution information, and a grant position 
coordinate of the structure. An example of label information is shown in Table 1. 
[0041] 
[Table 1] 
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(3 0 0, 5 0 0) 


1 0 


AtfJl/ 


(4 5 0, 2 5 0) 


1 0 




(1 5 0, 2 0 0) 


1 2 



The label information preparing part 22 will pass label information to the center control section 
24, if it finishes creating label information. The center control section 24 will order to transmit 
label information to the terminal 10 to the communications control 23, if label information is 
received. (Step 35). 

[0042]Label information is passed to the label information outputting part 17 of the terminal 10, 
and the label information outputting part 17 superimposes a name or attribution information of 
a structure in label information on a position in a landscape image (Step 36), A landscape 
image on which it was superimposed is displayed on graphic display devices, such as a 
display (CRT) and a head MARAN toddy spray (Step 37). An example of a landscape image in 
which it was superimposed on label information is shown in drawing 11 . 
[0043]lf an automobile rally game is started between a user of the terminal 10, and a user of 
other terminals, the communication control part 15 will send position information with time and 
car information of a car which a user of the terminal 1 0 operates to the center 20 for every 
fixed time with position information with time on the terminal 10 (Step 38). Through the center 
control section 24, position information with time and car information of a car of the terminal 10 
are passed to the game information Management Department 25 of the center 20, and the 
game information Management Department 25, Each user's position information with time and 
car information are managed, precedence information in a rally of each car which a user who 
participates in an automobile rally game drives is managed, and a user's position information 
and car information are passed to the center control section 24 (Step 39). A user's position 
information and car information are passed to the label information preparing part 22 by the 
center control section 24, and in the label information preparing part 22. When a registration 
number of the car has been recognized from a number plate of the car, a car which exists in a 
landscape image based on a user's position information and car information under rally game 
intervention, If the registration number can presume a registration number of a car which a 
certain user drives from a point of position information and car information, rally ranking label 
information on which a field of a car in a landscape image was made to superimpose 
precedence information of a rally game will be created, and the center control section 24 will 
be passed (Step 40). 
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[0044]The center control section 24 transmits rally ranking label information to the terminal 10 
(Step 41). The label information outputting part 17 of the terminal 10 displays rally ranking 
label information on the windshield of a car (Step 42). 
[0045] 

[Effect of the lnvention]Since according to this invention the geographical information on a 
computer and each portion in the landscape image of a real landscape can be matched and a 
user can be shown, as explained above, Human being compares the map and real landscape 
on a computer, and it does not match at human beings, ** also ends, and an automobile rally 
game is made in the actual street, and the existence and ranking understand only a rally 
participating vehicle in a national highway etc. 



[Translation done.] 
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